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Abstract —As a distributed hash table (DHT), Skip Graph 
acts as the underlying routing infrastructure of peer-to-peer 
(P2P) storage systems, distributed online social networks, search 
engines, and other DHT-based applications. For many P2P 
applications, data aggregation is vital, however, it is a missing 
feature of Skip Graph. The traditional aggregation algorithms 
cost noticeable message overhead which degrades the energy 
efficiency while increasing the response time. Likewise, the ag¬ 
gregation trees proposed for other DHTs are either inapplicable 
to the Skip Graph or apply some sort of randomness in their 
construction. Randomized features of an aggregation tree result 
in higher aggregation latency as well as enforcing unbalanced 
load on nodes which negatively affect the energy efficiency. In 
this paper, we propose ELATS which is the first energy and 
locality aware aggregation tree for Skip Graph. We define the 
energy awareness as minimizing the average energy cost of 
an aggregation tree, and the locality awareness as minimizing 
the latency on the path between the root and leaves of the 
aggregation tree. Performance analysis results show that ELATS 
algorithm provides both energy and locality awareness, and 
improves the aggregation latency with the gain of about 8% in 
comparison to the best existing solutions for DHTs which are 
either locality aware or energy aware. 

I. Introduction 

Skip Graph [1] is a prefix-based routing DHT where 
nodes are connected to each other based on their identifiers’ 
common prefixes. Fast searching [1], low traffic [2], high 
guarantee [2], scalability, and load balancing features of Skip 
Graph make it a suitable routing approach for P2P cloud 
services [3]—[8]. Likewise, Skip Graph can be used as an 
efficient underlying stmcture in applications ranging from 
distributed storage systems [9], to online social networks 
[10], and search engines [11]. 

Nodes in P2P systems employ data aggregation to obtain 
an identical view about a certain parameter of the system 
which is referred as the parameter of interest. Examples 
for parameter of interest in P2P systems are size of system 
[12], average available bandwidth, the reputation of a data 
object [13], and local clock [14]. Data aggregation is defined 
as repeatedly obtaining data about the parameter of interest 
from several peers, compressing or reducing the size of 
obtained data to achieve a smaller sized aggregated data, 
and forwarding the aggregated data to other peers of the 
system which causes all peers converge to a similar or 
identical view of the parameter of interest. Skip Graph 
suffers from the lack of a data aggregation capability. The 
traditional distributed data aggregation solutions are based 
on forwarding aggregated data to a subset of nodes (i.e. 
gossiping [15]—[17]), passing the aggregated data one by one 


(i.e. random walk [ 18]—[20]), and forwarding the aggregated 
data to all nodes (i.e. flooding). The traditional solutions, 
with the drawback of high message overhead, result in high 
aggregation latency and response time. Likewise, discarding 
the constraint on the number of exchanged messages as well 
as the size of the messages, the traditional solutions cause 
additional energy cost for the system. 

The existing DHT-based solutions aim at constructing an 
aggregation tree on top of the underlying routing infrastruc¬ 
ture, where the aggregation is done in two phases: request 
and reply. In the request phase, the root node initiates the 
aggregation request message and sends it to its children. 
The aggregation request message is forwarded in a top-down 
manner by each node to its corresponding children set until it 
reaches the leaves. Once a leaf node receives the aggregation 
request, it starts the reply phase by sending its state of the 
parameter of interest to its parent. Each node combines its 
parameter’s state with all aggregated states obtained from its 
children and sends it to its parent. The reply phase continues 
in a bottom-up manner until it reaches the root node. The root 
node generates the final result of aggregation by combining 
the aggregated states obtained from its children. If all nodes 
need to be notified about the aggregation result, the root node 
sends the result in a top-down manner similar to the request 
phase. 

The existing DHT-based aggregation trees are constructed 
based on the specific features of the system. For example [21] 
proposes an aggregation tree for Chord based on its circular 
property. So far, no aggregation tree is constructed specif¬ 
ically for Skip Graph. Likewise, the existing aggregation 
tree schemes for other prefix-based DHTs mainly focused on 
response time and does not consider the energy cost enforced 
by aggregation tree to the peers. 

In this study, we propose ELATS, the first energy and 
locality aware aggregation tree for Skip Graph. We define 
the locality awareness of an aggregation tree as minimizing 
the latency on the path between the root and the leaves. 
This improves the performance of aggregation tree in term 
of response time. We define the energy awareness as mini¬ 
mizing the average energy cost of the non-leaf nodes in the 
aggregation tree. The original contributions of this paper are 
as follows: 

• We propose ELATS which is the first fully decentralized 
energy and locality aware aggregation tree approach for 
the Skip Graph. ELATS is a fully decentralized recursive 
algorithm with each recursion executed on a single node. 


• We define the energy cost model of a generic aggrega¬ 
tion tree, and extend the Skip Graph simulator SkipSim 
[22] to support the aggregation tree models. 

• We redesign the proposed aggregation tree algorithms 
for other DHTs to be applicable on the Skip Graph, 
model them in the SkipSim and compare their perfor¬ 
mance with EL ATS. 

• Our simulation results show that compared to the 
best existing solutions which are either locality aware 
or energy aware, ELATS provides both the energy 
and locality awareness, and improves the aggregation 
latency with the gains of about 8%. 

II. Aggregation Tree: Energy Cost Model 

In our view of an aggregation tree, a node consumes energy 
while it exchanges request/reply messages, or aggregates the 
obtained data from its children [23]. All the aggregated data 
have the identical size which implies identical size of all the 
exchanged messages. We assume that exchanging a single 
message between two nodes in the aggregation tree charges 
an energy cost of E units on each of them. 

Energy Cost of Request Phase: The root node r initiates 
request phase by sending an aggregation request to its chil¬ 
dren. Equation 1 indicates the energy cost of the root node 
in the request phase, where the children set of r is denoted 
by CS r . 

E reqU est{r) = |<?S r | x E (1) 

During the request phase, an intermediate node i receives 
the request message from its parent, and forwards the re¬ 
quest to its children. Equation 2 shows the energy cost of 
intermediate node i during the request phase. 

E re quest(i ) = E + \CSi | X E (2) 

Equation 3 represents the average energy cost of non¬ 
leaf nodes in the aggregation tree T during the request 
phase, denoted by E request . It corresponds to the set of 
all intermediate nodes of T. 

j^ _ E reques t(r) + E request (i) 
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Energy Cost of Reply Phase: During the reply phase, an 
intermediate node i receives the aggregated states of nodes 
covered by sub-tree rooted at node i, adds its state of the 
parameter of interest to the set of obtained aggregated states, 
combines the states in the set, and sends the combined 
aggregated states to its parent. Equation 4 shows the energy 
cost of an intermediate node i during the reply phase. U 
denotes the energy cost of combining one pair of aggregated 
states. We assume that the combination is done linearly. The 
first two aggregated states are combined, and the result is 
iteratively combined with the next aggregated states until 
no aggregated state remains. Doing so, combining |C'<S'i| 
aggregated states as well as node’s i state of parameter of 
interest is done in ICS 1 ,;! steps, which costs the energy cost 
of \CSi\xU. 

E reply (i ) = flCSil x E) + flCSjl xU) + E (4) 

In the last step of the reply phase, the root node, r, receives 
the aggregated states of all nodes in the subtrees rooted at its 
children, adds its state of the parameter of interest to the set 


of obtained aggregated states, and combines the aggregated 
states. This results in the aggregated state of all nodes in the 
aggregation tree. Equation 5 shows the energy cost of the 
root node during the reply phase. 

E rep iy{r) = (|CS,.| x E) + {\CS r \ x U) (5) 
Equation 6 shows the average energy cost of non-leaf 
nodes in the aggregation tree T in the reply phase, denoted 
by E rep iy. 
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III. Energy and Locality Aware Aggregation 
Tree for Skip Graph (ELATS) 

In a Skip Graph, each node has two identifiers, a non¬ 
negative integer called numerical ID, and a binary string 
called name ID. In a Skip Graph with n nodes the search 
for name ID operation returns address the owns the most 
similar name ID to the search target by traversing (9(log n). 
The similarity of a pair of name IDs is defined as the length 
of their common prefix. ELATS works on the top of a locality 
aware Skip Graph where the pairwise latencies of nodes 
correspond to the length of their name IDs’ common prefix 
[7], A longer prefix corresponds to the lower latency. In our 
view of an aggregation tree, all nodes in the Skip Graph are 
spanned by the aggregation tree. The node starts aggregation 
process by invoking ELATS is named the initiator which 
becomes the root of resulted aggregation tree. We define a 
sub-domain as the set of name IDs with a common prefix. 
The common prefix represents a sub-domain called its sub¬ 
problem. For example, in a Skip Graph with name ID size 
of 4 bits, corresponding sub-domain to the 01 sub-problem 
is {0100,0101,0110,0111}. 

ELATS is a recursive distributed algorithm where each re¬ 
cursion is executed on a single node. The node who executes 
a recursion of ELATS is called executor. ELATS is shown by 
Algorithm 1. As the inputs ELATS receives the address of the 
executor node as a pointer, subProblem which is a binary 
string defines the sub-problem, energy cost of current path 
denoted by E c , latency of current path denoted by L c , and the 
number of intermediate nodes on the current path denoted by 
inernum. The current path is defined as the path between the 
executor and root of aggregation tree. In ELATS , the latency 
of the current path is obtained by summing up the common 
prefix lengths of consecutive nodes on the path. Likewise, the 
energy cost current path corresponds to the average energy 
cost of non-leaf nodes on the path between the executor and 
root of aggregation tree. 

ELATS first extracts the name ID of the executor denoted 
by namelD (Algorithm 1, line 1). The corresponding sub- 
domain of the subProblem. denoted by subDomain consists 
of the set of name IDs which start with the subProblem 
prefix. All name IDs in subDomain are as long as the 
executor’s name ID denoted by namelD. Likewise, all 
name IDs in subDomain. start with subProblem prefix of 
size \subProblem\ bits, and they differ only in their next 
\nameID\ — \subProblem\ bits. Hence, size of subDomain 



which corresponds to its number of name IDs is obtained as 
2 1 namelD\ — \subProblem | ( , ine 2 ). A SubDomain of size 1 

represents the situation where namelD is the only member 
of subDomain. In such a case, ELATS terminates since no 
parent-child relationship can be defined for one node (lines 
3 and 4). 

The core of ELATS is to decide on executor to be either 
the parent of all the subDomain name IDs, or the parent 
at most half of them. We call the former as covering all 
case, and the latter as covering half case. In covering half 
case, only the name IDs have more than \subProblem\ bits 
common prefix with namelD can become the children of 
executor. Moving from \subProblem\ to \subProblem\ + 1 
ignores half of the subDomain name IDs i.e., the ones 
who have exactly \subProblem\ bits common prefix. Since 
Skip Graph is locality aware, the remaining name IDs - 
which have at least \subProblem\ +1 common prefix length 
with the namelD- constitute the expected lower latency 
half portion of subDomain. To make this decision, ELATS 
computes the affected energy cost and latency of current 
path affected by each of cases (line 5). The energy cost of 
current branch of aggregation tree in the covering all and 
covering half cases are illustrated by E a and Eh, respectively. 
Each of these energy costs are obtained as the sum of the 
corresponding E request and E rep i y obtained from Equations 
3 and 6, respectively. In both equations \It\ = inernum. 

Equation 7 represents the latency of the current branch in 
the covering all case denoted by L a . The latency of current 
branch is defined as the minimum common prefix length of 
namelD with the name IDs in subDomain which is equal 
to \subProblem\. This minimum common prefix corresponds 
to the maximum latency between the executor and nodes 
of subDomain. If the executor becomes the parent of all 
subDomain name IDs, all of the name IDs in subDomain 
become leaves of the aggregation tree. The latency of the 
current branch is hence increased by the maximum latency 
between the executor and name IDs of subDomain as shown 
in Equation 7. In the covering half case, since the minimum 
common prefix length of namelD and the new sub-domain 
is \subProblem\ + 1, the corresponding latency denoted by 
Lh is obtained in a similar way shown by Equation 8. 

L a = L c + \nameID\ — \subProblem\ (7) 

Lh = L c + \nameID\ — ( \subProblem\ + 1) (8) 

After computing the energy cost and latency of current 
branch of aggregation tree in both covering all and covering 
half cases, ELATS computes the latency and energy cost 
improvements of covering all compared to covering half 
cases denoted by impL and impE , respectively (lines 6 
and 7). impL and impE are floating point values between 
0 and 1, and provided by the improvement function. On 
receiving two input arguments, the improvement function 
divides their absolute value of difference over the greatest 
input value, and outputs the result. 

A smaller than the e value of impp or impE conveys that 
going deeper than the current depth does not provide lower 
latency or better energy cost. This makes ELATS to choose 


Algorithm 1: ELATS 

Input: pointer executor, String subProblem, double E c , 
integer L c , int inernum 

1 namelD = e*ecMtor.getNamelD(); 

2 subDomainSize = ^ nameID ^ aubProblfirn ' i \ 

3 it subDomainSize == 1 then 

4 | terminate; 

5 computing L a , L h , E a , E h \ 

6 impL = improvement(L a , Lh)', 

7 impE = improvement^ a , Eh)', 

8 if impL < e or impE < e then 

9 avnodes = extend {subDomain)', 

10 for i £ avnodes do 

n parent(i) = executor', 

12 children(e:recutor).add(i); 

13 else 

14 if commonBits(subProblem + , 0 / , namelD ) == 

subProblem j then 

is a = searchByNamelD(sM6ProWem +'0'); 

16 if NULL then 

17 parent(a) = executor', 

is children(ea:ecutor).add(a:); 

19 a.ELATS(a, subProblem +'0', Eh , Lh, 

inernum + 1); 

20 ELATS(ea:ecMfor, subProblem +'l', Eh , Lh, 

inernum + 1); 

21 else 

22 a = searchByNamelD(subProblem +'l'); 

23 if a ■=/= NULL then 

24 parent(a) = executor', 

25 children(ea:ecutor).add(a:); 

26 a.ELATS(a, subProblem +' 1 ', Eh, Lh, 

inernum)', 

27 ELATS(ea:ecMfor, subProblem + , 0 , 1 Eh , Lh, 

inernum)', 

the covering all case, e is a close to zero local constant of ex¬ 
ecutor determined based on its properties like bandwidth and 
computational power. To perform covering all case, ELATS 
first finds all available name IDs in subDomain. A name ID 
is available if it has been assigned to an existing node in the 
Skip Graph. To find the available name IDs, ELATS employs 
the extend function which applies a searchForNamelD for 
each of the name IDs in subDomain, and checks their 
existence. The set of all available nodes of subDomain is 
denoted by avnodes. ELATS adds each node in avnodes to 
the children set of the executor, as well as makes the executor 
as the parent of all nodes in avnodes (lines 8-12). 

Large improvements in both impp and impE promise to 
meet better energy cost and lower latency as going deeper. In 
this case, the executor node needs to divide the subDomain 
into two smaller sub-domains and choose covering half 
case (line 13). namelD starts with the subProblem pre¬ 
fix of size | sub Problem). The \subProblem\ + 1 th bit of 
namelD is either 0 or 1. If the \subProblem\ + 1 th bit 
of namelD is equal to 1, the common prefix length of 
subProblem- 1-'O' and namelD computed by commonBits 
function is equal to \subProblem\ (line 14). For example, 
if namelD = 01010011000 and subProblem = 0101001, 
then subProblem +' O' = 01010010, and the common prefix 
of subProblem+' O' and namelD is 0101001 which is equal 



to the subProblem itself. In this case, the executor node 
should deliver the aggregation tree covering the sub-domain 
corresponds to subProblem+' 0' to another node which has 
the name ID prefix of subProblem+' O'. The executor node 
performs a searchForNamelD to find address of node a 
has the subProblem +' 0' prefix in its name ID (line 15). 

If node a exists, the executor node introduces itself as the 
parent of a and adds a to its children set (Algorithm 1, lines 
16-18). Likewise, ELATS calls a fresh recursion of itself on 
the node a for the subProblem+' O'. In the fresh recursion 
of ELATS, the current average energy cost of intermediate 
nodes on the path between the root of aggregation tree and 
node a is equal to the already computed Eh . Likewise, the 
latency of path from root node to a is equal to the already 
computed L h . Since the executor is the parent of a in the 
aggregation tree, the number of intermediate nodes on the 
path between the root node and node a is one more than the 
number of intermediate nodes on the path between the root 
node and executor (line 19). 

ELATS also calls a fresh recursion on the executor node 
for the subProblem +' 1' and with the similar average 
energy cost and latency arguments. However, since executor 
node runs the fresh recursion, no intermediate node is added 
(line 20). Lines 21-27 show the similar procedure for the 
case where \subProblem\ + 1 th bit of the namelD is equal 
to 0. Assuming that the initiator of aggregation tree is node 
/?, the initial call to ELATS is /3.ELATS(I3, NULL , 0, 0,1). 

Time Complexity: As shown by lines 2-4 of Algorithm 
1, a sub-problem that is as long as the name ID size of Skip 
Graph terminates the recursion of ELATS. The total number 
ELATS' s recursions on a single node is therefore bounded 
by the name ID size of Skip Graph. In a Skip Graph with n 
nodes, the name ID size is [ log n] . Hence, ELATS enforces 
(X log n) recursions on each node of the Skip Graph. The 
recursions of ELATS are executed in parallel on different 
nodes. 

IV. Related Works 

Table I provides a comparison between the decentral¬ 
ized aggregation tree schemes and our proposed ELATS. A 
prior work [21] proposes a parent function for the circular 
namespaces like Chord [9] where each node covers all its 
predecessors. Given a node’s identifier, the parent function 
returns back the parent’s identifier for that node. However, 
the parent function is not applicable to the Skip Graph which 
does not follow a circular continuous namespace. 

Applying a distributed breadth first search (BFS) [24] on 
the network’s graph results in a spanning BFS-tree which 
can be used as an aggregation tree. A BFS-tree guarantees the 
minimum number of intermediate nodes between each pair of 
nodes which implicitly results in a shorter aggregation tree. 
A short aggregation tree helps saving energy by reducing 
the number of exchanged messages. However, the locality 
awareness of the resulted aggregation tree is completely 
ignored in the BFS-tree. 


Strategy 

Energy Aware 

Locality Aware 

Parent Function [21] 

No 

Yes 

BFS-trees [24] 

Yes 

No 

SP-trees [25] 

No 

Yes 

Broadcast Tree [2], [21], [26], [27] 

Yes 

No 

Prefix-based Tree [26], [29], [30] 

No 

Yes 

ELATS 

Yes 

Yes 


TABLE I: Comparison of decentralized aggregation trees 


An aggregation tree has two degrees of freedom: nodes’ 
degree, and tree’s height. [25] claims the average nodes’ 
degree in a shortest path-based (SP-based) tree drops as the 
system size scales up, and converges to the degree of 3. Also, 
the degree of a node in this approach does not go beyond 
8. An SP-tree provides locality awareness. However, limiting 
nodes’ degree makes the aggregation tree to grow in height as 
the system size scales up. This results in increasing number of 
exchanged messages during the aggregation which negatively 
affects the energy cost of the tree. In a spanning broadcast 
tree [2], [21], [26], [27] each node broadcasts the received 
message to all its neighbors. The broadcasting procedure 
continues in a top-down manner until the message reaches the 
leaves. A broadcast tree can be used as an aggregation tree 
by broadcasting the aggregation request from the root node. 
Each node broadcasts the request to its children, aggregates 
and replies their responses back to its parent. 

Similar to Skip Graph, Kademlia [28] is a prefix-based 
routing DHT. There are many prefix-based broadcast trees 
[26], [29], [30] proposed for Kademlia. In a prefix-based 
broadcast tree, the system is divided into disjoint regions, 
where each region covers a specific prefix. A broadcast 
initiator initiates the broadcast by contacting one node per 
region. 

V. Simulation Results 

We extended the Skip Graph simulator SkipSim [22] and 
developed models to simulate the aggregation tree algorithms. 
We implemented ELATS, broadcast tree, and prefix-based 
tree in SkipSim, and compared their performance in terms 
of energy and locality awareness. Each simulation consists 
of 100 random topologies. In our simulations, we consider 
E = 8 J, U = 1J, and for ELATS e = 0.01. 

Figure 1 shows the average energy cost of non-leaf nodes 
of the aggregation tree as the system capacity scales up. 
The system capacity is defined as the maximum number of 
registered users to the system. As the system capacity scales 
up exponentially the average number of children assigned 
to non-leaf nodes in prefix-based aggregation tree increases 
exponentially. This results in an exponentially increasing 
energy cost on non-leaf nodes of the prefix-based tree. The 
degree of each node in the broadcast tree is bounded by the 
number of its neighbors. Having a system capacity of n, the 
number of neighbors of a node in the Skip Graph is (9(log n). 
Hence the exponentially increasing system capacity does not 
have a noticeable effect on the energy cost of the broadcast 
tree. Similar to the broadcast tree, ELATS acts independent of 
the system capacity in terms of energy cost and provides the 
similar energy cost as the broadcast tree which is the most 
energy efficient existing solution. 
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Fig. 1: Average energy cost on parent nodes in the aggregation tree vs System 
capacity. Y-axis shows the base 10 logarithms of the average energy cost. 
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Fig. 2: The average of maximum root to leaves’ path latency in the aggregation tree 
vs System size. Y-axis shows the base 10 logarithms of average path latency. 

Figure 2 shows the average of the maximum latency 
of root to leaves’ path in aggregation trees as the system 
capacity scales up. In a locality aware Skip Graph as the 
system capacity increases, the system is divided into more 
condensed regions and more accurate locality aware name 
IDs are achieved [7]. This results in an improvement in the 
latencies of prefix-based and ELATS. Since the broadcast 
tree, which performs as the best energy efficient existing 
solution, is not locality aware, it performs the worst in terms 
of latency when compared to ELATS and prefix-based tree. 
As shown in Figure 2, in comparison to the prefix-based 
which acts as the best existing locality aware solution, 
ELATS improves the average value of the maximum root to 
leaves’ path latency with the gain of about 8%. 


VI. Conclusion 

To improve the energy efficiency and response time of the 
aggregation procedure in a Skip Graph, we propose ELATS , 
the first fully decentralized energy and locality aware aggre¬ 
gation tree. We simulated and compared ELATS with state- 
of-the-art aggregation tree algorithms for DHTs. Compared 
to the best existing solutions which are either locality aware 
or energy aware, ELATS improves aggregation latency with 
a gain of about 8% while simultaneously providing similar 
energy efficiency to the best energy aware existing solution. 
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